Project 2: Image Homography and Warping¶

Part 1: Image Homography Estimation¶

Scene 1¶

B Rectifed to A¶

C Rectified to A¶

Scene 2¶

A Rectifed to B¶

The homography transformation was applied to remove perspective distortion from the angled images. This led to each tilted surface appearing fronto-parallel, as if the photo were taken straight on. For each scene, four corner points were manually selected on the planar region of interest, and a 3×3 homography matrix H was estimated using the Direct Linear Transform (DLT) algorithm. Applying inverse warping with H⁻¹ produced the rectified image, restoring the surface’s true rectangular proportions.

Homography works by defining a precise mathematical mapping between two planar views under perspective projection. Because the poster and sign surfaces in these test scenes are flat, their perspective distortions can be corrected perfectly. This technique is foundational in computer vision tasks such as document scanning, augmented-reality overlays, planar surface alignment, and texture mapping where accurate geometric consistency between real and virtual planes is critical.

Part 2: Creative Application (Exploration and Extension)¶

For this part, I used a poster hanging on my wall to create an augmented-reality effect through homography. After manually selecting four corner points on the new scene, a second image was warped to match the poster’s perspective so that it appeared naturally anchored within the photo. This demonstrates how a planar homography can accurately project digital content onto real-world surfaces, creating the illusion that the overlay is part of the environment.

What worked best was how precisely the transformation aligned the overlay once the corner points were refined. The edges followed the scene’s perspective almost perfectly, and the illusion of depth felt convincing. The main challenges were lighting differences that made the overlay stand out and coordinate drift when the clicked points weren’t perfectly consistent. Balancing geometric accuracy and visual realism required several refinements, but the result clearly shows how a single homography matrix can fuse digital imagery onto a real surface as if it were physically printed there.

Part 3: Warping Comparison¶

Two image-warping techniques were applied to the same set of facial landmarks to evaluate how each method handles localized and global deformations. Both methods used identical control points but approached the deformation process differently.

• Triangular Mesh Warping applies local affine transformations within each triangle formed by the control points. This gives it fine-grained control over specific regions of the image, making it fast and predictable. However, because each triangle is transformed independently, visible seams or discontinuities can appear where adjacent triangles fail to align perfectly. This effect is most noticeable around sharp features such as the nose or jawline, where abrupt local differences can make the warp look slightly segmented or “cut.” Despite that limitation, it remains efficient and performs well in real-time systems where localized edits or animations are needed.

• Thin-Plate Spline (TPS), by contrast, treats the entire deformation as a smooth, continuous function by minimizing bending energy across the image. Rather than transforming triangles individually, TPS models the deformation globally, ensuring that motion at one landmark gradually influences nearby areas. This produces smoother and more natural-looking results, especially for organic surfaces such as human faces. The trade-off is that TPS is computationally more expensive and can occasionally over-smooth regions that require sharper control, like corners or edges.

In practice, the triangular mesh warp tends to preserve structure and sharpness at the cost of continuity, whereas TPS excels in producing visually coherent and realistic transformations across the entire surface. For applications like facial animation or shape morphing, TPS is generally preferred for its smoothness and fluid motion, while the triangular mesh approach remains a strong choice for interactive systems, AR filters, or localized adjustments that don’t require full-frame smoothing.

In this project, the difference between the two methods was clear: the triangular mesh warp introduced small discontinuities around high-contrast features, whereas the TPS deformation appeared smoother and more cohesive. TPS produced a more natural transformation that resembled real facial movement, while the triangular mesh felt stiffer but more controlled. Together, these results highlight the trade-off between computational speed and visual realism in non-rigid image warping.